Using the Eggplant DAI Runner
The Eggplant DAI Runner lets you run your pre-configured tests that were created in the Test Configuration tab and view the results of those tests via color coded dots. You can also schedule test runs from the Runner tab and see the tests that are in progress. To access the Runner tab, click Controller on the left menu of the Eggplant DAI interface.
The Runner page in Eggplant DAI
Running Tests
To run a test, from the Runner tab, click the 
Run button for the test config that you want to run. Repeat this step if you need to run more tests.
Remember to re-upload any scripts that you may have changed manually via Eggplant Functional or in the Eggplant DAI Snippet Editor back to the Execution Environment (Manage Agents > Manage Suites) before running the test, or your changes may be overwritten.
Note that when executed through a parent model, submodels will use the same suite as the parent model.
To stop a test configuration instance that's already in progress, click 
.
When you abort a test run that is part of a test config schedule, all runs that are part of that test config schedule are aborted.
If a test run is initiated (either at a scheduled time or by clicking the Run button) while another test run for the same test config is in progress, then the newly-started test run immediately completes with an error.
The Runner page displays the following information about your test configs:
-
Test Configuration: The names of your test configs. This column can be sorted alphabetically.
-
Model/Suite: A link to the Eggplant DAI model or suite used for the selected test config, each indicated by their respective icons to the left - where a building block
represents a model and a folder 
represents suites. -
Schedule Type: The type of schedule for the test config. Displays No Schedule if none exists for a given test configuration. See Scheduling Options below for a list of schedule types.
-
Scheduled Start: The date and time that a scheduled test run will start for a given configuration. This field is blank if no schedule exists for that test config.
-
Previous Results: Shows the results of the ten most recent test runs (if applicable) represented via color-coded dots (described below). Hovering on an individual dot shows the status and date/time of the runs. Click to expand to see a snapshot of the results of that run.
Indicates that the test was successful.
Indicates that the test has failed.
Indicates that the test configuration is in progress.
Indicates that the test resulted in error.
Indicates that the test run was aborted.
See Viewing Execution Results for more information.
- Last Executed: The date and time of the most recent run. Displays In Progress if a test run is ongoing. This column can be sorted chronologically.
Run Selected Steps
The Run selected steps feature allows you to execute specific steps within a test configuration (test config) instead of running the entire test. You can watch an example of how to use the Run selected steps panel in DAI here.
To use this feature, click the 
Action menu for a test config you want to run a subset of test steps from, and select Run selected steps. A slider panel opens where you can select the steps to include or exclude in a run.
The Run selected steps slider panel for a test config
Filter Options
The Run selected steps panel enables you to filter the test steps you want to include or exclude in your test config run. The filter options available depend on the type of test config you selected:
-
Model-based test configs have 3 filter options.
-
Script-based test configs have 4 filter options.
The filter options include:
- Step status
- Test case tag
- Test step name
- Scenario tag (only available for script-based test configs)
The steps you select automatically display in the Test steps to run section on the right side of the Run selected steps panel, along with the latest step status for each step displayed. The step status refers to the last time that step was run from within this test config. Individual test cases or steps of a test config may be marked as Untested if some steps were moved, added, or removed from the test config since the last run. A specific test case or test config step will be marked as Skipped if it is excluded by filtering during a test run.
The last run status of each step is determined by looking at the last run of the step within the current test configuration, not the last run of the test case. For example, if a test case passed the last time the current test configuration was executed, but has since failed when run as part of another test config (or as a Live Run), then we consider the last run status to still be Passed within the context of the current test config.
Please be aware that any test step statuses from before version 25.2.0 show a Untested status until they are run again. This is due to significant differences in the structure of test configuration information between versions.
Run the Selected Test Steps
When you finish selecting steps, click Run to run the selected test steps.
If a test is currently running for the selected test config, the Run selected steps button remains disabled until the test finishes.
Quick Run
The Quick run button enables you to quickly rerun steps. You can see this button by clicking any of the result dots under Previous Results for a test config.
The following quick run options are available:
-
Failed Steps → Runs selected steps with the Failed status pre-selected.
-
Errored Steps → Runs selected steps with the Error status pre-selected.
-
Failed & Errored Steps → Runs selected steps with both Failed and Error statuses pre-selected.
Like the Run selected steps button, the Quick run button will be disabled while a test configuration is running.
Viewing Execution Results
Accessing Detailed Test Execution Results: In the Previous Results column, you can gain insights into your test config execution by clicking on the individual run result icons. These icons represent various outcomes such as Pass,
Fail,
In progress,
Error or
Canceled.
Clicking on them will expand a detailed summary of the test execution.
Test execution summary view Within this summary, you'll find following information:
- Execution result: This indicates whether the test was successful or encountered issues.
- Run details: Details such as when the execution started, its duration, and information about the System Under Test (SUT).
- Steps completed: The number of steps successfully executed.
- Total reruns completed: The total number of reruns performed across all steps.
- Total test runs completed: The overall test runs completed, including both initial runs and any reruns.
- Steps ignored: The number of steps ignored in the test configuration.
- Max run time: The time period after which to cancel a test config run.
Donut chart A donut chart provides a visual representation of your test configuration's results. The chart showcases the following:
- Passed first time: The number of test cases that passed on their initial attempt.
- Passed after rerun: Test cases that initially failed but ultimately passed after a rerun.
- Failed: Test cases that did not meet the success criteria.
- Errored: Test cases that encountered errors during execution.
- Cancelled: Test cases that were incomplete.
- In progress: Test cases that are currently running.
Please note that in cases involving multiple reruns, the donut chart will display only the final result of the rerun. To learn more about the re-run feature visit the following page: Test configuration
Test Configuration Logs For a more in-depth understanding of the testing process and detailed test case execution results, you can refer to the Test Configuration Logs. These logs provide a record of the testing and relevant details.
Clicking the Pass, Fail, In Progress dots shows the summary of the test execution
Click Open in results to drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution, but you can switch to the default All Runs view by clicking View all results link at the top of the display. Note that the Open in results button is available for all test results except for the runs that went into Error state.
Clicking the Error execution dot shows the reason for the error in the expanded view:
Click to view the expanded summary
Deleting a Test Run from the Runner
In the Previous Results column, click the individual color-coded dots representing Success, Failure, In Progress , Error, Cancelled to view the summary of that test execution in an expanded view.
In the expanded view, click Open in Results to further drill down into the results of the selected test execution. This opens the Test Results tab showing all the individual test runs for the selected test execution.
At the top right corner of the screen is a red button labeled as Delete Run. Clicking on it will render a confirmation box. Click on Confirm to proceed with the deletion.
The instance of the selected test execution as well as all of its results will then be deleted.
Once deletion is completed, the screen will redirect back to Runner tab.
Scheduling Test Runs
Scheduling adds great flexibility to your test runs, enabling you to set your tests to run at specific intervals with the frequency you require.
If a test execution is initiated (either at a scheduled time or by clicking the Run button) while another test execution for the same test configuration is in progress, then the newly-started test execution immediately completes with an error.
Test Schedule Type Run Once
Scheduling Options
Use the Test config schedule page shown above to set schedules for your test config runs. You can display the Test config schedule page using one of the following two methods:
- Display the Runner page (Controller > Runner), locate the test configuration you want to create a schedule for, and click its specified Schedule Type. (The default schedule type is No schedule.)
- Display the Runner page, locate the test configuration you want to create a schedule for, click its Actions menu, and select Edit.
DAI provides the following schedule types:
| Schedule Type | Description |
|---|---|
| No Schedule (default) | Runs are not scheduled for this test config. You do not see the calendar date or time selectors on the Test config schedule page for this option. |
| Run once at selected time | Runs a test once at the selected date and time. Selecting this option displays the calendar date and time selectors shown above. |
| Run on selected then repeat every | Runs a test at the selected date, time, and frequency. This option displays the calendar and time selectors, and a Repeat every field with a drop-down list shown below where you can select Days (default), Minutes, Hours, or Weeks. Selecting this option runs the test for the first time at the selected date and time, and then repeatedly according to the frequency you specify.  |
| Custom | Runs a test at a specified date and time, and provides the option to use Cron syntax in the Custom time interval field shown below. Selecting this option runs the test for the first time at the selected date and time, and then repeatedly according to the Cron syntax schedule you specify. DAI Cron syntax recognizes Monday as the first day of the week instead of Sunday as shown below in Custom Cron Schedule Examples.  |
Custom Cron Schedule Examples
# schedule executing at midnight from Monday to Friday \{#schedule-executing-at-midnight-from-monday-to-friday}
0 0 * * 0-4
# schedule executing every weekday at 5:30PM \{#schedule-executing-every-weekday-at-530pm}
30 17 * * 0-4
For information about using Cron syntax, click the help (?) button next to the Custom time interval field. You can also see this article.
If you are not familiar with using Cron syntax but would like to learn more, please contact our Customer Support.
Step-by-Step: Scheduling Tests
-
Display the Test config schedule page using one of the following two methods:
- Display the Runner page (Controller > Runner), locate the test configuration you want to create a schedule for, and click its specified Schedule Type. (The default schedule type is No schedule.)
- Display the Runner page, locate the test configuration you want to create a schedule for, click its Actions menu
..., and select Edit.
-
Select the radio button for the scheduling option you want:
- Run once at selected time
- Run on selected time and repeat every
- Custom
The Test config schedule page displays the scheduling option you selected and the Start date and time calendar date and time selectors.
- If you choose Run on selected and the repeat every, you also see the Repeat every field and interval selector shown above in Scheduling Options.
- If you choose Custom, you also see the Custom time interval field where you can enter Cron syntax as shown above in Scheduling Options.
-
Choose the date and time you want the test config to execute or execute for the first time, and specify the time interval or Cron syntax if appropriate for your selection.
-
Click Save to save your schedule or Cancel to discard it.
Viewing All Runs
You can view all runs within a test configuration, including any executions that didn't go through, runs that are in progress, and typical runs with usual pass or fail status. Viewing all runs within a test configuration is particularly useful to locate the details to determine the likely cause of a failed run.
Viewing Run History
From the Actions menu on the test config you want to see the history of, select View all runs. The results open in a separate interface displaying the following information with the most recent runs:
- Execution Date: The date and time of the test configuration.
- Status: The status of the test runs in color-coded capsule format. The options are as follows:
- Success
- Failed
- In Progress
- Error
- Canceled
- Open in Results: Click Open in Results to further drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution, but you can switch to the default All Test Runs view by clicking View all results.
- Expand row button: Click to view a snapshot of the results of the selected execution.
Click the Runner navigation link to switch back to the Runner tab.
Deleting Test Runs from Run History
Click Open in Results to further drill down into the results of the selected test execution. This opens the Results tab showing all the individual test runs for the selected test execution.
At the top right corner of the screen is a red button labeled as Delete Run. Clicking on it will render a confirmation box. Click on Confirm to proceed with the deletion.
The instance of the selected test execution as well as all of its results will then be deleted.
Once deletion is completed, the screen will redirect back to Run History tab.
Set Retry Limits and Intervals for Connections to SUTs and Execution Environments
You can configure the DAI Runner to automatically retry failed test connections to a SUT or Execution Environment with the following environment variables:
-
RETRY_SLEEP_INTERVAL_SECONDS -
RETRY_TIMEOUT_SECONDS
The DAI Server configuration file includes these variables by default. See Retries the connection for SUT and Execution Environments for explanations of these variables and instructions for modifying your DAI Server configuration file.